System, method and apparatus for remote monitoring

ABSTRACT

A monitoring unit for security and automation in a premises are described. The monitoring unit uses remote configuration and control to enable monitoring of a premises. The monitoring unit provides multiple monitoring functions to respond to events within the space and alert a user at a remote device like a smartphone. An image sensor provides a wide field of view which can be segmented on the mobile device to enable specific areas to be viewed enabling the user to view the area and be alerted when an event occurs based upon a rules based configuration.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/866,521, filed May 4, 2020, which is a continuation of U.S.application Ser. No. 14/456,449, filed Aug. 11, 2014, which claims thebenefit of U.S. Provisional Application No. 61/864,248, filed Aug. 9,2013. All of these prior applications are incorporated by reference intheir entirety.

TECHNICAL FIELD

The present disclosure relates to monitoring systems and in particularto a web-based integrated monitoring system for small spaces.

BACKGROUND

Home security and monitoring systems require professional installationand require professional management and monitoring. For small spacessuch as apartments, the installation of a monitoring system is notpractical due to the investment required to install the system and theon-going expense to monitor and maintain the monitoring system. Inaddition, in the rental apartment market, landlords do not want theadditional expense of a monitoring system and renters do not want toinstall as system as they cannot take it with them. Existing solutionsrequire multiple components to be installed and provide limitedcontrollability and monitoring capability. Accordingly, an improvedmonitoring system remains highly desirable.

INCORPORATION BY REFERENCE

Each patent, patent application, and/or publication mentioned in thisspecification is herein incorporated by reference in its entirety to thesame extent as if each individual patent, patent application, and/orpublication was specifically and individually indicated to beincorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1F show views of a monitoring unit, under an embodiment.

FIG. 2A is a system overview of the monitoring system, under anembodiment.

FIG. 2B is a system overview of the monitoring system 200A, under analternative embodiment.

FIG. 3 is a hardware block diagram of the monitoring system, under anembodiment.

FIG. 4 is a method of operation of the monitoring unit, under anembodiment.

FIG. 5 shows a block diagram of the monitoring unit firmware, under anembodiment.

FIGS. 6A-6B is an example monitoring system dashboard user interface ona mobile device, under an embodiment.

FIG. 7 is an example monitoring system night stand mode user interfaceon a mobile device, under an embodiment.

FIG. 8 is an example monitoring system status screen on a mobile device,under an embodiment.

FIG. 9 is an example live streaming user interface display on a mobiledevice, under an embodiment.

FIG. 10 is an example live streaming multi-view user interface displayon a mobile device, under an embodiment.

FIG. 11 shows monitoring system accessory control on a live streaminguser interface on a mobile device, under an embodiment.

FIGS. 12A-12C show control and scheduling screens for the monitoringunit on a mobile device, under an embodiment.

FIGS. 13A-13B show a user interface for configuring rules for themonitoring system unit on a mobile device, under an embodiment.

FIG. 14A is a front perspective view of the monitoring unit with adetachable stand, under an embodiment.

FIG. 14B is a rear perspective view of the monitoring unit with adetachable stand, under an embodiment.

FIG. 15A is a front perspective view of the monitoring unit with adetachable wall bracket, under an embodiment.

FIG. 15B is a rear perspective view of the monitoring unit with adetachable wall bracket, under an embodiment.

DETAILED DESCRIPTION

Embodiments of a monitoring system are described herein. The monitoringsystem is configured for use in smaller spaces, for example, but is notso limited. The architecture of the monitoring system includes numerouslayers that operate in concert to provide security, notification, videostreaming, and home automation functions, as described in detail herein.

FIGS. 1A-1F show views of a monitoring unit 100, under an embodiment.FIG. 1A shows a front view of the monitoring unit 100. The monitoringunit 100 integrates multiple devices and/or operations into a compactform factor to provide monitoring functions. The monitoring unit 100comprises a lens (e.g., fish-eye lens, etc.) with a camera 102 thatoperates to view a wide target area from a single location. A motionsensor 104 is included to detect motion within the target area. Themonitoring unit 100 of an embodiment includes one or more environmentalsensors for monitoring parameters of the local environment. For example,the monitoring unit 100 includes an ambient light sensor 106 and/or atemperature and humidity sensor 114. The monitoring unit 100 alsoincludes an indicator 108 (e.g., LED indicator, etc.) to provide visualstatus on the operation of the monitoring unit 100. A microphone 110, aspeaker 112, and a siren 116 are also included to detect noises in theenvironment, provide feedback, allow two way communications and alertnoises. A stand 118 is coupled or connected to the monitoring unit 100but may be removed for wall mounting applications.

FIG. 1B is a perspective view of the monitoring unit 100, under anembodiment. FIG. 1C is a rear view of the monitoring unit 100, under anembodiment. FIG. 1D is a side view of the monitoring unit 100, under anembodiment. FIG. 1E is a top view of the monitoring unit 100, under anembodiment. FIG. 1F is a bottom view of the monitoring unit 100, underan embodiment.

FIG. 2A is a system overview of the monitoring system 200, under anembodiment. The monitoring system 200 includes the monitoring unit 100along with one or more additional components as appropriate to aninstallation of the monitoring unit 100. For example, the monitoringunit 100 of an embodiment is coupled to a wide area network (e.g., theInternet) via a Wi-Fi router and/or cellular data coupling orconnection. The system also includes an application for installation ona user's smartphone or tablet, and a web application accessible from anycomputer. Additionally, the system 200 includes a cloud-based back-endsupporting the mobile application, web application, and device firmware.

FIG. 2B is a system overview of the monitoring system 200A, under analternative embodiment. The monitoring system 200A includes a pluralityof monitoring units 100-1/100-N (collectively referred to as 100) alongwith one or more additional components as appropriate to an installationof the monitoring units 100. For example, the monitoring units 100 of anembodiment are coupled to a wide area network (e.g., the Internet) via aWi-Fi router and/or cellular data coupling or connection. The systemalso includes an application for installation on a user's smartphone ortablet, and a web application accessible from any computer.Additionally, the system 200A includes a cloud-based back-end supportingthe mobile application, web application, and device firmware of themonitoring units 100. The monitoring system 200A comprising a pluralityof monitoring units is described in detail herein.

The monitoring unit is installed in a user's home (e.g., on the wall,using the provided wall-mount, on a flat surface, etc.) and is poweredby a power source. The power source of an embodiment includes a DirectCurrent (DC) wall adapter, for example, but is not so limited.Battery-backup is available to ensure the security aspects of the systemremain functional if there is a loss of power. The monitoring unit of anembodiment uses encryption for all outgoing and incoming data. A usercan set up multiple monitoring units to monitor and secure severalareas.

Users interact with the environment in which a monitoring unit isinstalled in a number of ways. The monitoring unit's mobile and webapplications show current and historical environmental readings of itssurroundings by reporting on temperature, humidity, ambient light andsound. This information is periodically uploaded by monitoring unit andstored in the cloud infrastructure using a time-series database. It ispresented using current values and graphs in the Vitals section of themobile and web application.

The monitoring unit's internal sensors provide a wealth of informationor data about the device and its surroundings, for use in security andnotification scenarios. The data provided includes but is not limited toinformation representing one or more of the following: unexpected motionwithin monitoring unit's field of view (FOV); temperature changes withinthe space; humidity changes within the space; physical movement ofmonitoring unit (due to vibrations or tampering); loud, unexpectedsounds; and changes in ambient light.

The monitoring unit of an embodiment can be coupled or connected tovarious remote or peripheral devices or sensors. The monitoring unitincludes a home area network (HAN) radio, or personal area network(PAN), to control and receive information from paired accessories suchas sensors, switches, and dimmers. HAN-connected accessories can becontrolled by way of security rules, a programmable schedule andinternal sensor triggers such as ambient light and temperature. The HANdevices include but are not limited to IEEE 802.11 Wireless Local AreaNetwork devices, and IEEE 802.15 Wireless Personal Area Network devices,for example Wi-Fi, Zigbee and/or Z-wave based devices, but are not solimited.

The monitoring unit includes couplings or connections among a variety ofremote components like remote sensors and other devices at the premises,and supports discovery, installation and configuration of the remotedevices coupled or connected to the system, as described in detailherein. The monitoring unit uses this self-generated sub-network todiscover and manage the remote devices at the premises. The monitoringunit thus enables or forms a separate wireless and/or wired network, orsub-network, that includes some number of devices and is coupled orconnected to the LAN or WAN of the host premises. The monitoring unitsub-network can include, but is not limited to, any number of otherdevices like wired devices, wireless devices, sensors, cameras,actuators, interactive devices, WiFi devices, and security devices toname a few. The monitoring unit manages or controls the sub-networkseparately or privately from other communications and transfers data andinformation between components of the sub-network and the LAN and/orWAN, but is not so limited.

The monitoring unit also provides a coupling or connection to a centralmonitoring station (CMS) data for remote monitoring of the premises. Thedata of an embodiment is provided to the CMS and to the remote device,but in other embodiments is provided to one of the remote device and theCMS. Under this embodiment, one or more monitoring units are coupled tothe CMS via a network (e.g., one or more of WiFi, LAN, WAN, cellular,etc.). Alternatively, the monitoring units are coupled to the CMS viathe network (e.g., WAN, cellular, etc.) and an intermediate server ordevice (e.g., remote server, etc.). In operation, the monitoring unittransmits collected data and information to the CMS based upon auser-selected state of the monitoring unit. The data transmitted by themonitoring unit includes data of the monitoring unit as well as data ofand data received from devices coupled to the monitoring unit via thelocal sub-network. The monitoring unit automatically delivers data ofone or more onboard and/or coupled devices to the CMS. The interactionsand notifications between the monitoring unit and the remote CMS of anembodiment are controlled or managed by the mobile application runningon the mobile device. As such, the user interface presented by themobile application provides controls for enabling or disabling remotemonitoring by the CMS; for example, a user can activate monitoring atthe CMS via the mobile application when leaving town on a trip, and candeactivate CMS monitoring upon his/her return. The monitoring unit andremote server therefore provide a mechanism to activate and deactivatemonitoring by the remote CMS.

An embodiment of this mechanism is an Application Programming Interface(API) using an interface technology such as REST or SOAP, for example,to send monitoring activation and deactivation messages to the CMS andto receive acknowledgements from the CMS. Other embodiments such as theclient application, monitoring device, or remote server utilize the userselection to enable/disable the delivery of activity messages to theCMS, where the CMS is always available and uses the presence of messagesto trigger monitoring periods. The current invention also anticipatesthe integration of the CMS billing system into the service to enableon-demand billing of monitoring services, and/or to offer time-basedmonitoring of the system (e.g. the CMS monitoring is active for aspecific period of time).

Users can place the monitoring unit into one of a plurality of securitymodes (e.g., Home, Away, Vacation, Off) using the mobile application,thereby activating and deactivating the various security preferences(defined as rules). Other security modes, such as ‘CMS Monitoring mode’for example, can be utilized as well to effectuate different behaviorsfor the device and/or for its monitoring. Rules and other configurationsmay be stored on the monitoring unit's firmware and as such do notrequire a centralized server environment. In another embodiment theserules and configurations are stored on a remote server or backed up to aremote server to facilitate replacement of a defective unit.

The monitoring unit's mobile application allows users to set rules foreach security mode pertaining to notifications, home-automation actionsand alarms based on a set of scenarios. Under a scenario, the monitoringunit's various sensors (both internal and externally paired) can alert auser to activity within their environment, using data from sensors. Thenotification options of an embodiment include but are not limited tomobile push, SMS messages, telephone calls, and electronic mail to namebut a few.

Under another scenario, sensors and their corresponding actions areconfigured by way of the mobile application. The monitoring unit canalso leverage the use of externally paired HAN sensors to drive actionsand notifications. The HAN sensors can include one or more ofthermostats, door sensors, actuators, door locks, garage openers, windowsensors, light dimmers or switches, to name a few.

The monitoring unit under yet another scenario allows rules associatedwith sensors (whether externally-paired or internal) to controlconnected appliances by way of paired HAN dimmers and switches.Furthermore, the monitoring unit can control the state of HAN-connectedappliances by way of a configurable schedule, based on time and/orsunrise/sunset based on installed location.

The monitoring unit allows a user to set up notification-only rules thatare outside the scope of any security modes. These rules can result inmobile push notifications derived from the same sensors that triggersecurity mode rules.

The monitoring unit can alert the surrounding environment to a potentialbreach of security by way of a very loud siren, driven by rulesassociated with sensors, both internal and externally paired. The sirencan also be triggered by external parties such as the CMS and/or theuser from a remote device. This capability allows a remote entity tointeract with the device to warn occupants or deter an intruder.Moreover, the monitoring unit's system receives weather and otherenvironmental information. This can influence rules and also provideadditional environmental status to the mobile and web applications.

The monitoring unit allows users to connect to a live stream of videoand audio via their mobile application or any remote device. This videois captured and streamed using a very wide field-of-view (FOV) lens,allowing the user to electronically pan, tilt, and zoom within theirspace. Additionally, multiple angles of the captured live stream can beviewed at once, in a segmented fashion. Each segment represents adistinct view of the monitoring unit's surroundings and the directionand zoom level chosen by the user are retained when a user returns tothe live stream.

Conventional video cameras using a wide angle lens and local on-camera‘de-warping’ removed the distortion imparted by the wide angle lenslocally on the camera processor to produce a flattened image, and thenstreamed portions or all of the flattened image to a remote device. Inthese systems the remote device displayed the de-warped video, but hadno ability to simulate the raw video data being presented by the lens.These conventional systems therefore were optimized for lower-end remotedevices that were not capable of advanced video processing.

In contrast to these conventional camera technologies, the monitoringunit described herein comprises ‘Immersive 3D video streaming’, whichtransmits lens-warped video data collected at the camera to the remotedevice where it is de-warped. In an embodiment, the raw lens-warpedvideo data collected at the camera is transmitted or streamed to theremote device in a highly compressed format; in various alternativeembodiments the warped video data collected at the camera can be clippedor processed in some manner before being compressed and transmitted orstreamed. Regardless of any pre-processing technique applied to thevideo data collected at the camera, the embodiments described hereintransmit or stream warped video data from the monitoring unit to theremote device, and the remote device performs de-warping of the videodata. However, other alternative embodiments can de-warp the video dataat the monitoring unit prior to transmitting the data stream to a remotedevice.

The local processor of the remote device manipulates the received videodata to provide an optimal user experience. A key distinction in thisapproach is the ability to rely upon the high performance video decodingand three-dimensional (3D) manipulation capabilities present in state ofthe art remote devices, which include but are not limited to smartphones, tablet computers, personal computers, and other mobile and/orportable processor-based devices. Generally, the Immersive 3D videostreaming process executing on the remote device decodes and decryptsthe video stream to a raw video frame buffer, creates a 3D space thatemulates the specific lens geometry of the camera, and maps the videoframe buffer to the 3D space providing an ‘immersive 3D video view’ thatallows the remote device to zoom, pan, and move around in the 3D spacegiving the perception of being ‘inside the lens looking around’.

The monitoring unit of an embodiment generates a Immersive 3D videostream using components comprising a lens with a wide-angle geometry, asdescribed in detail herein, that ‘warps’ or distorts the video to obtainthe wide-angle view. The monitoring unit includes an image encoder thatencodes the video image into a compressed streaming format. Themonitoring unit of an embodiment stores the compressed streaming formatwith warped video to a local storage device coupled to the monitoringunit. Alternatively, the monitoring unit stores the compressed streamingformat with warped video at a remote server or other remote processingcomponent or memory to which it is coupled via a network coupling (e.g.,LAN, WAN, Internet, etc.). Alternative embodiments may use other devices(e.g., a local Digital Video Recorder-DVR, etc.) to accomplish theprocess of encoding, compression, and storage separately from themonitoring device itself.

The monitoring unit streams the compressed video to a remote device(e.g., smart phones, tablet computers, personal computers, other mobileand/or portable processor-based devices, etc.). The monitoring unit ofan embodiment streams the compressed video directly to the remotedevice. Alternatively, however, the monitoring unit streams thecompressed video to the remote device via an intermediate server (e.g.,relay server, intermediate DVR, etc.).

The remote device decompresses the received compressed video stream. Theremote device decompresses the video stream and then further processesthe resulting decompressed video images using data of the camera lensgeometry, more specifically the wide-angle geometry (de-warping) of thecamera lens. For example, the remote device of an embodimentdecompresses the video stream using a software codec (e.g. FFMPEG)executing on a processor. The remote device of an alternative embodimentdecompresses the video stream using a hardware codec. The remote deviceuses 3D rendering technology to map the warped video to a 3D spacereplicating the lens geometry. The data of the lens geometry used by theremote device to process the received video stream is used by the mobileapplication, under an embodiment, and is one or more of receiveddynamically from a remote server or monitoring unit, included in amapping table at the mobile device, and known a priori, but is not solimited. In an alternative embodiment the lens geometry is specified aspart of the data interchange associated with the video feed setup. Inyet another alternative embodiment the mobile application stores data ofa plurality of known lens geometries associated with the camera typessupported by the application.

The remote device ‘maps’ the decompressed warped image to the 3D spacerepresenting the lens geometry and displays this 3D view using a displaythat is a component of or coupled to the remote device. The remotedevice includes a user interface that enables a user to ‘move’ aroundthe environment of the monitoring unit by panning and zooming around the3D space and the mapped video image. The user interface of an embodimentis generated by the mobile application, but is not so limited. The userinterface of an embodiment enables a user to navigate the 3D space usingpinching gestures and swiping gestures when the remote device includes atouchscreen display. Additionally, the remote device enables playing ofand interacting with stored video clips generated by the monitoringunit, where the stored video clips are stored at local storage or remoteserver in the same way.

By way of example in an embodiment, the process for a remote device toreceive the Immersive 3D video stream from the monitoring unit comprisesthe remote device creating a tunnel (e.g., Secure Sockets Layer (SSL),etc.) to the monitoring unit by coupling or connecting to an externalport that was configured by the monitoring unit (e.g., using UniversalPlug and Play (UPnP), etc.). The monitoring unit encodes the raw videoimage (e.g., 1280×1070) using an encoder application (e.g., H.264 withHigh Profile, etc.). The monitoring unit sends the encoded video streamto the mobile device using the tunnel (e.g., sends video stream usingKIP tunneled in RTSP).

The mobile device of this example embodiment includes a multimedia datalibrary (e.g., FFmpeg library, etc.) that decodes packets of the videostream (e.g., Real-time Transport Protocol (RTP) packets of H.264stream) to an image buffer (e.g., a YUV color space image) in memory.The size of the memory buffer of an example is 1280×1070, but is not solimited. The mobile device creates an a virtual surface (e.g., OpenGraphics Library (OpenGL)) through an API for rendering vector graphics.The virtual surface of this example embodiment is YUV for rendering thepan/zoomed image, where the image size is based on the mobile device,but the embodiment is not so limited. The mobile device user interfaceincludes controls (e.g., pinch with fingers, zoom with fingers, etc.)for selecting a position of the rendered image on the display of themobile device. Based on the selected position on the image, the mobiledevice takes a portion of the decoded image (e.g., YUV) and executes ade-warping and scaling algorithm (e.g., OpenGL) to produce a renderedsubset image into the image surface.

Users interact with their premises during a live-streaming session bycontrolling appliances and speaking into their space through themonitoring unit's built-in two-way audio functionality, which streamslive audio from the mobile application to the monitoring unit's speaker.The monitoring unit supports two-way voice sessions under numerousembodiments. For example, a remote device of an embodiment initiates atwo-way voice session with one or more monitoring units at a premises.Similarly, a third-party monitoring station initiates a two-way voicesession with one or more monitoring units at a premises. Additionally,the monitoring unit provides live video contemporaneously with or as acomponent of the two-way voice session with the remote device and/orthird party monitoring station. The two-way voice sessions includesessions over a WAN (e.g., Internet Protocol (IP) via WiFi, etc.) and/orsessions over a cellular network (e.g., cellular voice, IP data, etc.),but are not so limited.

The monitoring unit can record video of events associated with rulestriggered by internal and externally paired sensors. The monitoring unitof an embodiment continuously records video and audio in a loop,enabling it to report on an event by presenting footage before and afterit occurs. Users can review these recorded events on the mobile and webapplications and perform electronic pan, tilt, and zoom operationswithin the captured video, as though they were streaming it in realtime.

The monitoring unit records a longer video if subsequent events happenin rapid succession. A single video is created which encapsulates theseveral events in question. The system understands that this particularvideo maps to several events, and vice-versa.

The monitoring unit allows users to record video in an on-demandfashion, triggered from the mobile and web applications. As withevent-driven footage, users can perform electronic pan, tilt, and zoomoperations within the captured on-demand video recordings via the mobileand web applications. The monitoring unit also includes smart sounddetection, allowing captured loud sounds to be fed to characterizationsoftware which helps identify it, (e.g. a smoke alarm, barking dog,etc.).

The monitoring unit periodically transmits a heartbeat signal or messageto the cloud or other network infrastructure, allowing the user to benotified when the monitoring unit disconnects from the Internet. Theuser is also notified when the monitoring unit reconnects and continuesto post heartbeat information. If connectivity issues arise and eventstrigger video recording, the monitoring unit saves videos locally andqueues them for later upload, when connectivity resumes.

The cloud infrastructure of an embodiment comprises one or more of anumber of components. The components include for example, front-end webservices the expose or include an Application Programming Interface(API) for both the mobile application and the monitoring unit firmware.This public traffic is encrypted and authentication is performed againststrong firmware and user credentials. Additionally, back-end databasesinclude user account information and settings, and monitoring unitconfiguration and time-series sensor and event data. Back-end in-memorydatabases house real-time sensor history, cached data and heartbeatinformation.

The cloud infrastructure components also include notification servicesand worker services. The notification services send information to usersby way of direct and third-party assisted methods. These include email,mobile push notifications, SMS and voice calls. The worker servicesprocess uploaded content, check in-memory real-time data for connecteddevices' heartbeats, trigger notifications, start on-demand recordingsfor users and perform additional infrastructure and product-relatedfunctions. File storage services providing fast and reliable disk spacefor the entire infrastructure.

Additionally, the cloud infrastructure components include infrastructurebackup services, multi-site disaster recovery for database, worker andweb services, and redundancy for each component. High availability forall worker and web services is provided by way of application-levelclustering and load balancing for incoming web service requests frommobile apps and firmware.

Multiple monitoring units are installed at various independent locationsat a premises, and when so installed function as a distributed premisessecurity system, under an embodiment and as described herein. Themonitoring units of this collective installation automatically discoverand couple to each other and share data over a device network (e.g., IP,WiFi, wired connection, Z-Wave, etc.) that is separate and independentfrom a LAN and/or WAN to which the monitoring units are coupled at thepremises. In an embodiment the monitoring units utilize the LAN (Wife,Ethernet, etc.) to couple to each other and share data. In anotherembodiment the monitoring units utilize a WAN (such as a cellular orbroadband network) to couple to each other and share data. In yetanother embodiment the monitoring devices are installed in physicallyremote locations (such as a home and an office) and are coupled via aWAN, but can still share data and form a distributed security network.The monitoring units thereby combine logically to form an integratedmonitoring or security network at the premises or between premises. Eachmonitoring unit includes an automatic installation process for addingand removing itself from this integrated network. The monitoring unitsare configured to repeat at least one message between devices in theintegrated security network formed by the coupling of the devices. Whenthe installation includes multiple monitoring units, the collection ofmonitoring units are controlled or monitored from a single mobileapplication and are associated and managed with a single user account.Similarly, the collection of monitoring units is coupled to a remoteservice and monitored at the CMS.

FIG. 3 is a hardware block diagram of the monitoring system, under anembodiment. The monitoring unit's hardware architecture comprises but isnot limited to one or more of the following components: ARMSystem-on-chip 302; DDR Memory 304; Flash storage 306; Home area networkRF module with antenna 308; Wi-Fi (or local area network technology)module with antenna 310; Cellular data module with antenna 312 ifprovisioned; Camera system comprising a multi-megapixel CMOS sensor andvery wide FOV lens 314; Audio system comprising a microphone 316 andspeaker 318; Alarm siren 320; Passive infrared (PIR) motion sensor witha very wide FOV Fresnel lens 322; Temperature sensor 324; Relativehumidity sensor 326; Accelerometer 328; Ambient light sensor 330; Powersystem 332 comprising a DC jack 334 and battery compartment 336; ROB LEDindicator 338; Push-button 340.

FIG. 4 is a method of operation of the monitoring unit, under anembodiment. The monitoring unit's firmware is based upon an operatingsystem such as Linux, for example, but is not so limited. Specializedsoftware or applications along with this operating system provides theservices, API and functionality for set up and use of the monitoringunit's features in concert with the cloud infrastructure and mobile andweb applications.

During the user's initial setup of monitoring unit, the following tasksare performed by the firmware:

-   -   a. The monitoring unit's firmware boots.    -   b. Since no existing device information is present, the        monitoring unit creates a Wi-Fi access point for setup        functions.    -   c. User launches the mobile application and after creating an        account using their information begins the setup process.    -   d. User connects to monitoring unit's Wi-Fi access point and        submits Wi-Fi credentials for their home network.    -   e. The monitoring unit attempts to connect with the home network        using the provided Wi-Fi credentials.    -   f. The monitoring unit registers itself to the cloud back-end,        associates with the current user and attempts to open ports on        the user's Internet router (for incoming connections) using        Universal Plug and Play (UPNP) or Network Address Translation        (NAP) Port Mapping Protocol (PMP), depending on the type of        router present.    -   g. Once fully connected, the monitoring unit turns off its Wi-Fi        access point and begins normal operation.    -   h. In the cases where a new Wi-Fi router is present, the        monitoring unit has moved to a new environment, or connectivity        to the existing router fails, the monitoring unit can accept new        Wi-Fi credentials in a similar fashion to the initial setup        process.

Embodiments described herein include a setup or enrollment process thatcomprises determining geolocation of the monitoring unit duringinstallation at the premises. The monitoring unit of an embodimentincorporates a WiFi module (processor and radio (802.11)), and duringenrollment the monitoring unit puts the WiFi module into ‘Access Pointmode’. The mobile device running the mobile application described indetail herein enrolls as a WiFi client to the monitoring unit accesspoint. The mobile application then provides new WiFi credentials (e.g.,service set identification (SSID), password (optional), etc.) to themonitoring unit via the WiFi access point; subsequently, the mobileapplication automatically switches the mobile device over to the sameWiFi SSID, or the user manually switches the mobile device to that SSIDusing a network configuration utility. Upon receipt of the new WiFicredentials, the monitoring unit automatically switches its WiFiprocessor to enroll as a client at the new WiFi SSID (using the optionalpassword). Either the monitoring unit or the mobile applicationinitiates a process to store the WiFi credentials on a remote server orother remote device. The monitoring unit of an embodiment restores theWiFi credentials from a remote server, but the remote server of analternative embodiment initiates restoration of the WiFi credentials ofthe monitoring unit.

The mobile application of an embodiment provides numerous operations,but is not so limited. For example, the mobile application provides auser interface that enables a user to switch the monitoring unit to theaccess point mode in order to change the SSID. The mobile applicationprovides authentication directly to the camera (e.g. username, password,etc.). Alternatively, the mobile application provides authenticationagainst a remote server.

The mobile application provides to one or more monitoring units locationinformation corresponding to the monitoring unit installation, where thelocation information corresponding to the monitoring unit is locationdata determined at the mobile device. The monitoring unit then providesits location data to the remote server. Alternatively, the mobileapplication provides the location data of the monitoring unitinstallation directly to a remote server or other remote device. Themonitoring unit of an embodiment includes an administrative tool thatprovides information about numerous monitoring units and theirrespective physical locations.

In an alternative embodiment the monitoring unit is temporarily coupledor connected via a physical connector (e.g. a USB cable) to a mobiledevice running the mobile application. In this embodiment the mobileapplication delivers the Wifi SSID and password over the wiredconnection, and the monitoring device then switches to the Wifi accesspoint as described above.

Generally, the monitoring unit's operating state comprises but is notlimited to the following:

-   -   a. Sensor polling is running and receiving raw data from        sensors.    -   b. The rules engine is running and can interface with sensors.    -   c. The audio and video service and RTSP server are running and        are ready to accept incoming connections, record footage in a        loop and detect loud sounds.    -   d. The PIR motion sensor service is running and able to detect        movement within the monitoring unit's FOV.    -   e. Automated tasks run at their pre-defined intervals and        perform, but are not limited to, one or more of the following:        maintain contact or communication between the monitoring unit        and the cloud back-end and ensure incoming ports remain open on        the user's Internet router; check for updates to the monitoring        unit's firmware; post status updates about the current        environment around the monitoring unit; post heartbeats        periodically to inform the cloud backend of the monitoring        unit's state.

Sensors and Rules

The sensor polling service reads from internal sensors (e.g.,temperature, humidity, ambient light, acceleration/motion, etc.) andsends the data to the rules engine. It can also receive a signal fromany other part of the firmware to force an immediate read of thesensors. All sensor data is sent to the rules engine.

The PIR motion sensor service reads from the PIR software driverdirectly, but is not so limited. The motion sensor, which implements aBessel filter in order to eliminate false positives, issues a message tothe Rules engine if a threshold for motion is exceeded.

When loud sound above a predefined threshold is detected, a signal ispassed to the rules engine. When appropriate, the loud sound in questionis passed through characterization software to help identify it, (e.g. asmoke alarm, barking dog, etc.).

The rules engine loads a list of rules for notifications for the currentsecurity mode (home, away, or vacation) from a database into memory. Therules engine also loads any HAN control rules that can be controlled viaschedule, ambient light, temperature or any other sensors, be theyinternal or external. Notification-only rules are processed in parallelto mode-based security rules.

The rules engine saves the data with a timestamp in the monitoringunit's firmware database. The data is also sent to each activerule/control in order to determine what action, if any, should be taken(e.g. turn on an appliance, sound the siren, notify the user etc.).

Audio and Video

The audio and video service is responsible for streaming media, savingto a file and detecting loud sounds.

For saving footage to a file, audio and video are encoded and placedinto a circular buffer of a certain time. This enables the monitoringunit to capture video and audio “before” an event has occurred. Thisqueue is operating when the system is on but is not so limited.

For streaming, video and audio are encoded and served via RTP/RTSP to auser's mobile application. The streaming is encrypted and supportsmultiple clients at once.

FIG. 5 shows a block diagram of the monitoring unit firmware, under anembodiment. The mobile application is the user's interface to themonitoring unit. The mobile application is executed on a smartphone orother personal or mobile electronic device. Within the application, theuser's account is created, security and notification rules are defined,environmental readings are displayed, live streaming takes place andother settings are submitted to the cloud back-end and the monitoringunit's firmware. The application also serves as a tool to set up themonitoring unit's hardware, enabling the monitoring unit to pair withthe user's home Wi-Fi network.

Key functions are accessed from the application's tab bar at the bottomof the screen, once the user has logged into their account.

FIGS. 6A-6B show an example monitoring system dashboard user interfaceon a mobile device, under an embodiment. The Dashboard provides anat-a-glance view of the monitoring unit's system and provides access toone or more of the following functions: the monitoring unit's currentsecurity mode; the temperature near the monitoring unit; the monitoringunit's Wi-Fi signal strength; current status of HAN-connectedaccessories; weather alerts, if present; access to Events, Recordings,and Settings; access to the Night Stand.

FIG. 7 is an example monitoring system Night Stand mode user interfaceon a mobile device, under an embodiment. The user can activate the NightStand mode (available from the Dashboard), providing access to thevarious HAN-connected control accessories, a clock, weather information,and a panic button. Pressing the panic button activates the monitoringunit's siren.

FIG. 8 is an example monitoring system status screen on a mobile device,under an embodiment. The Vitals section displays monitoring unit'sinternal sensor readings and external weather information in aneasy-to-understand format. Historical information is displayed usinggraphs allowing the user to see trends for each reading.

FIG. 9 is an example live streaming user interface display on a mobiledevice, under an embodiment. Tapping or selecting the center icon of themobile application's tab bar launches live streaming and couples orconnects the user securely to the monitoring unit's built-in wide FOVcamera. The user can then pan, tilt, and zoom the live stream ofmonitoring unit's surroundings, control connected HAN plug-in modules(e.g. turning on a light) and stream audio from the microphone of theirmobile device to the user's space by way of monitoring unit's built-intwo-way audio feature.

FIG. 10 is an example live streaming multi-view user interface displayon a mobile device, under an embodiment. Users can also choose to switchto a multi-view version of the live stream, which allows them to look atdifferent areas of their space at the same time. This is achieved bypresenting the user with several smaller views of the pan, tilt, andzoom video. The video image is segmented allowing multiple areas withinthe field of view of the camera to be isolated. The image captured bythe camera has wide angle viewing area such as provided by a fish-eyelens. The image maybe de-warped and post-processed to provide a moreviewable image. The monitoring unit remembers the last-used settings ofthe live stream, including the direction the user was looking and thezoom level of the video.

FIG. 11 shows monitoring system accessory control on a live streaminguser interface on a mobile device, under an embodiment. If the user haspaired HAN control accessories to monitoring unit, they can be accessedand controlled from within the live video screen. This allows a user toturn an appliance on or off and see the results in real time, ifdesired.

FIGS. 12A-12C show control and scheduling screens for the monitoringunit on a mobile device, under an embodiment. The Controls sectionallows configuration and control of HAN-connected accessories. Theseaccessories can be configured via one or more of a timed schedule,sunrise/sunset, ambient light level, and temperature.

FIGS. 13A-13B show a user interface for configuring rules for themonitoring system unit on a mobile device, under an embodiment. Therules section allows the user to set security-related actions formotion, loud sound and temperature change triggers. Actions can be setfor each security mode and include but are not limited to one or more ofthe following: record video and audio of the event; notifications; pushmessage; electronic mail; phone call; SMS message; notification to auser's Trusted Circle members; the sounding of the monitoring unit'sbuilt-in siren; control of any connected HAN switches.

Additionally, notification-only options are present which allow the userto be informed of events outside the scope of the current security mode.

Additional functionality may be provided by the camera such as motiondetection and ambient light detection. The processor may use imageprocessing to determine characteristics of the image for use in motiondetection, face recognition or light detection. In addition themicrophone may be used for voice recognition function of the monitoringunit.

FIG. 14A is a front perspective view of the monitoring unit with adetachable stand, under an embodiment. FIG. 14B is a rear perspectiveview of the monitoring unit with a detachable stand, under anembodiment.

FIG. 15A is a front perspective view of the monitoring unit with adetachable wall bracket, under an embodiment. FIG. 15B is a rearperspective view of the monitoring unit with a detachable wall bracket,under an embodiment.

Embodiments described herein include a monitoring unit comprising acamera. The monitoring unit comprises a network interface. Themonitoring unit comprises a processor coupled to the camera and thenetwork interface. The monitoring unit comprises at least oneapplication executing on the processor. The processor receives an imagefrom the camera. The processor receives sensor data from at least onesensor coupled to the processor. The processor generates an alert basedupon a change in at least one of the image and the sensor data. Thealert is sent via the network interface to a mobile device.

Embodiments described herein include a monitoring unit comprising: acamera; a network interface; a processor coupled to the camera and thenetwork interface; and at least one application executing on theprocessor, wherein the processor receives an image from the camera,wherein the processor receives sensor data from at least one sensorcoupled to the processor, wherein the processor generates an alert basedupon a change in at least one of the image and the sensor data, whereinthe alert is sent via the network interface to a mobile device.

The monitoring unit of an embodiment comprises at least one a memorydevice coupled to the processor.

The monitoring unit of an embodiment comprises at least onecommunication module coupled to the processor.

The at least one communication module comprises a home area network(HAN) radio frequency (RF) module.

The at least one communication module comprises a Wi-Fi module.

The executing of the at least one application generates an enrollmentprocess.

The enrollment process automatically places the WiFi module into anAccess Point mode.

The mobile device comprises a mobile application, wherein the mobileapplication enrolls as a client to the Access Point.

The mobile application provides WiFi credentials to the processor viathe Access Point.

At least one of the mobile application and the processor initiatestorage of the WiFi credentials on a remote server.

At least one of the remote server and the processor restore the WiFicredentials from the remote storage device.

The mobile application provides authentication against at least one ofthe processor and a remote server.

The processor automatically switches the WiFi module to enroll as aclient using the WiFi credentials.

The mobile application automatically switches the mobile device toenroll using the WiFi credentials.

The mobile application provides a user interface that includes at leastone control for switching the processor to the Access Point mode tochange the WiFi credentials.

The mobile application provides to a device location informationcorresponding to installation of the monitoring unit.

The device comprises a remote server.

The device comprises at least one of the monitoring unit and at leastone additional monitoring unit.

The monitoring unit of an embodiment comprises an administrativeapplication that provides information about at least one monitoring unitthat includes the location information.

The at least one communication module comprises a local area network(LAN) module.

The at least one communication module comprises a cellular data module.

The at least one communication module is coupled to a remote device tocommunicate with the remote device.

The at least one communication module is coupled to a remote device tocommunicate over an Internet Protocol (IP) channel.

The at least one communication module is coupled to a remote device tocommunicate over a cellular channel.

The communication comprises a two-way voice session with the remotedevice.

The communication comprises a data session, wherein video images aretransmitted during the data session.

The remote device comprises the mobile device.

The remote device comprises a central monitoring station.

The communication module automatically establishes a coupling with theat least one sensor.

The communication module automatically establishes a coupling with alocal area network (LAN) at the premises.

The at least one application transfers data between at least one deviceon the LAN.

The communication module forms a sub-network at the premises.

The sub-network is a private network.

The at least one sensor is coupled to the sub-network.

Devices couple to the sub-network and communicate over the sub-network,wherein the devices include at least one of wireless devices, wireddevices, and IP devices.

The monitoring unit of an embodiment comprises a remote server includinga user account coupled to the processor.

The camera comprises an image sensor and a lens.

The camera comprises a lens including a wide-angle geometry, wherein thecamera generates images including warped images, wherein the cameragenerates the images using a wide-angle view mapped to the geometry ofthe lens from collected images.

The camera comprises an encoder that encodes collected images togenerate a processed data stream.

The processed data stream is a compressed video stream.

The monitoring unit of an embodiment comprises memory coupled to thecamera, wherein the camera stores to the memory the processed datastream that includes warped video.

The memory is local to the camera.

The memory is remote to the camera.

The camera streams the processed data stream to a remote device, whereinthe remote device comprises at least one of a mobile device and aserver.

The camera streams the processed data stream directly to the remotedevice.

The camera streams the processed data stream to the remote device via atleast one intermediary device.

Remote device processes the processed data stream using knowledge of awide-angle geometry of the lens.

The processing comprises decompressing the processed data stream.

The remote device comprises a software codec, wherein the software codecdecompresses the processed data stream.

The remote device comprises a hardware codec, wherein the hardware codecdecompresses the processed data stream.

The processing comprises using three-dimensional (3D) rendering andmapping warped video to a 3D space representing at least a portion ofthe lens geometry.

The processing comprises displaying a 3D view of the collected imagesvia a display coupled to the remote device.

The remote device comprises a user interface comprising control gesturesfor navigating around the 3D view presented via the display.

The navigating comprises at least one of panning and zooming around the3D view presented via the display, wherein the control gestures compriseat least one of pinching gestures and swiping gestures.

The camera comprises at least one of a video camera and an imagingcamera.

The camera comprises a CMOS sensor and very wide FOV lens.

The monitoring unit of an embodiment comprises an audio system.

The monitoring unit of an embodiment comprises an alarm siren.

The at least one sensor comprises a motion sensor.

The motion sensor comprises a passive infrared (PIR) motion sensor witha very wide FOV Fresnel lens.

The at least one sensor comprises an environmental sensor.

The environmental sensor comprises at least one of a temperature sensorand a humidity sensor.

The at least one sensor comprises an accelerometer.

The at least one sensor comprises an ambient light sensor.

The monitoring unit of an embodiment comprises a power system.

The monitoring unit of an embodiment comprises at least one indicatorcoupled to the processor.

The at least one application generates at least one notification.

The at least one notification comprises one or more of a push message,an electronic mail, a telephone call, a Short-Message-Service (SMS)message, a notification to at least one contact.

The monitoring unit is coupled to one or more accessories.

The accessories are controlled by at least one of a timed schedule, asunrise/sunset event, an ambient light level, and a temperature.

The monitoring unit of an embodiment comprises a rules engine executingon the processor.

The monitoring unit of an embodiment comprises a mobile applicationinstalled on the mobile device.

The mobile application generates a user interface presented on themobile device, wherein the user interface provides access to at leastone of the image and the sensor data.

The at least one application is at least one of accessed and controlledusing the user interface.

The at least one sensor is controlled via the user interface.

The monitoring unit of an embodiment comprises at least one actuatorcoupled to the processor, wherein the at least one actuator iscontrolled via the user interface.

The monitoring unit of an embodiment comprises a heartbeat signalgenerated by the processor and transmitted to a remote device.

The monitoring unit of an embodiment comprises at least one remoteserver coupled to the network interface.

The coupling comprises at least one of a wide area network and acellular network.

The at least one remote server comprises a central monitoring station.

The processor transmits to the central monitoring station at least oneof the image and the sensor data.

The processor transmits to the central monitoring station a messagecomprising information representing at least one of the image and thesensor data.

The mobile device comprises a mobile application.

The mobile application comprises an interface for enabling and disablingremote monitoring by the central monitoring station.

The mobile application comprises an interface for controllingcharacteristics of the message and transmission of the message.

Embodiments described herein include a monitoring unit comprising aplurality of sensors. The plurality of sensors includes an image sensor.The monitoring unit comprises a network interface. The monitoring unitcomprises a processor coupled to the plurality of sensors and thenetwork interface. The monitoring unit comprises at least oneapplication executing on the processor. The processor receives sensordata from the plurality of sensors. The processor generates an alertbased upon a change in the sensor data. The alert is sent via thenetwork interface to a mobile device associated with a user.

Embodiments described herein include a monitoring unit comprising: aplurality of sensors, wherein the plurality of sensors include an imagesensor; a network interface; a processor coupled to the plurality ofsensors and the network interface; and at least one applicationexecuting on the processor, wherein the processor receives sensor datafrom the plurality of sensors, wherein the processor generates an alertbased upon a change in the sensor data, wherein the alert is sent viathe network interface to a mobile device associated with a user.

Embodiments described herein include a system for remote monitoring. Thesystem comprises a monitoring unit at a premises. The monitoring unitcomprises a processor coupled to a plurality of sensors. The pluralityof sensors includes an image sensor. The processor includes at least oneapplication executing on the processor. The processor receives sensordata from the plurality of sensors and generates monitoring unit data.The system includes a server and a database located remote to thepremises and coupled to the monitoring unit via a network coupling. Theserver receives the sensor data and the monitoring unit data and storesthe sensor data and the monitoring unit data in the database. The serverprovides access to the sensor data and the monitoring unit data via amobile device.

Embodiments described herein include a system for remote monitoring, thesystem comprising: a monitoring unit at a premises, the monitoring unitcomprising a processor coupled to a plurality of sensors, wherein theplurality of sensors include an image sensor, wherein the processorincludes at least one application executing on the processor, whereinthe processor receives sensor data from the plurality of sensors andgenerates monitoring unit data; and a server and a database locatedremote to the premises and coupled to the monitoring unit via a networkcoupling, wherein the server receives the sensor data and the monitoringunit data and stores the sensor data and the monitoring unit data in thedatabase, wherein the server provides access to the sensor data and themonitoring unit data via a mobile device.

The processor generates an alert based upon a change in at least one ofthe sensor data and the monitoring unit data, wherein the alert is sentto the mobile device.

The system of an embodiment comprises at least one communication modulecoupled to the processor.

The at least one communication module comprises a home area network(HAN) radio frequency (RF) module.

The at least one communication module comprises a Wi-Fi module.

The executing of the at least one application generates an enrollmentprocess.

The enrollment process automatically places the WiFi module into anAccess Point mode.

The mobile device comprises a mobile application, wherein the mobileapplication enrolls as a client to the Access Point.

The mobile application provides WiFi credentials to the processor viathe Access Point.

At least one of the mobile application and the processor initiatestorage of the WiFi credentials on the server.

At least one of the server and the processor restore the WiFicredentials from the server.

The mobile application provides authentication against at least one ofthe processor and a server.

The processor automatically switches the WiFi module to enroll as aclient using the WiFi credentials.

The mobile application automatically switches the mobile device toenroll using the WiFi credentials.

The mobile application provides a user interface that includes at leastone control for switching the processor to the Access Point mode tochange the WiFi credentials.

The mobile application provides to a device location informationcorresponding to installation of the monitoring unit.

The device comprises the server.

The device comprises at least one of the monitoring unit and at leastone additional monitoring unit.

The system of an embodiment comprises an administrative application thatprovides information about at least one monitoring unit that includesthe location information.

The at least one communication module comprises a local area network(LAN) module.

The at least one communication module comprises a cellular data module.

The at least one communication module is coupled to a remote device tocommunicate with the remote device.

The at least one communication module is coupled to a remote device tocommunicate over an Internet Protocol (IP) channel.

The at least one communication module is coupled to a remote device tocommunicate over a cellular channel.

The communication comprises a two-way voice session with the remotedevice.

The communication comprises a data session, wherein video images aretransmitted during the data session.

The remote device comprises the mobile device.

The remote device comprises a central monitoring station.

The communication module automatically establishes a coupling with theplurality of sensors.

The communication module automatically establishes a coupling with alocal area network (LAN) at the premises.

The at least one application transfers data between at least one deviceon the LAN.

The communication module forms a sub-network at the premises.

The sub-network is a private network.

The plurality of sensors are coupled to the sub-network.

Devices couple to the sub-network and communicate over the sub-network,wherein the devices include at least one of wireless devices, wireddevices, and IP devices.

The server includes a user account.

The image sensor comprises a camera including a lens.

The camera comprises a lens including a wide-angle geometry, wherein thecamera generates images including warped images, wherein the cameragenerates the images using a wide-angle view mapped to the geometry ofthe lens from collected images.

The camera comprises an encoder that encodes collected images togenerate a processed data stream.

The processed data stream is a compressed video stream.

The system of an embodiment comprises memory coupled to the camera,wherein the camera stores to the memory the processed data stream thatincludes warped video.

The memory is local to the camera.

The memory is remote to the camera.

The camera streams the processed data stream to a remote device, whereinthe remote device comprises at least one of the mobile device and theserver.

The camera streams the processed data stream directly to the remotedevice.

The camera streams the processed data stream to the remote device via atleast one intermediary device.

Remote device processes the processed data stream using knowledge of awide-angle geometry of the lens.

The processing comprises decompressing the processed data stream.

The remote device comprises a software codec, wherein the software codecdecompresses the processed data stream.

The remote device comprises a hardware codec, wherein the hardware codecdecompresses the processed data stream.

The processing comprises using three-dimensional (3D) rendering andmapping warped video to a 3D space representing at least a portion ofthe lens geometry.

The processing comprises displaying a 3D view of the collected imagesvia a display coupled to the remote device.

The remote device comprises a user interface comprising control gesturesfor navigating around the 3D view presented via the display.

The navigating comprises at least one of panning and zooming around the3D view presented via the display, wherein the control gestures compriseat least one of pinching gestures and swiping gestures.

The camera comprises at least one of a video camera and an imagingcamera.

The camera comprises a CMOS sensor and very wide FOV lens.

The system of an embodiment comprises an audio system coupled to theprocessor.

The system of an embodiment comprises an alarm siren coupled to theprocessor.

The plurality of sensors comprises a motion sensor.

The motion sensor comprises a passive infrared (PIR) motion sensor witha very wide FOV Fresnel lens.

The plurality of sensors comprises an environmental sensor.

The environmental sensor comprises at least one of a temperature sensorand a humidity sensor.

The plurality of sensors comprises an accelerometer.

The plurality of sensors comprises an ambient light sensor.

The at least one application generates at least one notificationcorresponding to the alert.

The at least one notification comprises a notification to at least onecontact.

The at least one notification comprises one or more of a push message,an electronic mail, a telephone call, and a Short-Message-Service (SMS)message.

The monitoring unit is coupled to one or more accessories.

The accessories are controlled by the monitoring unit data.

The accessories are controlled by a schedule.

The system of an embodiment comprises a rules engine executing on theprocessor.

The system of an embodiment comprises a mobile application installed onthe mobile device.

The mobile application generates a user interface presented on themobile device, wherein the user interface provides access to at leastone of the image and the sensor data.

At least one of the server, the database, and the at least oneapplication are at least one of accessed and controlled using the userinterface.

The plurality of sensors are controlled via the user interface.

The system of an embodiment comprises at least one actuator coupled tothe processor, wherein the at least one actuator is controlled via theuser interface.

The system of an embodiment comprises a heartbeat signal generated bythe processor and transmitted to at least one of the server and themobile device.

The system of an embodiment comprises at least one remote server coupledto at least one of the monitoring unit, the server, and the mobiledevice.

The coupling comprises at least one of a wide area network and acellular network.

The at least one remote server comprises a central monitoring station.

The processor transmits to the central monitoring station the monitoringunit data.

The processor transmits to the central monitoring station a messagecomprising information representing the monitoring unit data.

The mobile device comprises a mobile application.

The mobile application comprises an interface for enabling and disablingremote monitoring by the central monitoring station.

The mobile application comprises an interface for controllingcharacteristics of the message and transmission of the message.

The system of an embodiment comprises at least one additional monitoringunit at the premises.

The at least one additional monitoring unit is physically separated atthe premises from the monitoring unit.

The at least one additional monitoring unit is coupled to the monitoringunit.

The coupling includes at least one of a wired coupling, wirelesscoupling, WiFi coupling, and IP coupling.

The system of an embodiment comprises forming an integrated securitynetwork at the premises by logically combining the at least oneadditional monitoring unit and the monitoring unit.

At least one of the monitoring unit and the at least one additionalmonitoring unit comprise an automatic installation process thatautomatically controls at least one of adding and removing a monitoringunit to the integrated network.

The system of an embodiment comprises a central monitoring stationcoupled to at least one of the server, the monitoring unit, and the atleast one additional monitoring unit.

The monitoring unit and the at least one additional monitoring unit aremonitored and controlled from the mobile device.

The monitoring unit and the at least one additional monitoring unit aremonitored and controlled from the mobile device.

The server comprises a user account that corresponds to the monitoringunit and the at least one additional monitoring unit.

Embodiments described herein include a system for remote monitoring. Thesystem comprises a monitoring unit at a premises. The monitoring unitcomprises a processor coupled to a camera and a network interface. Theprocessor includes at least one application executing on the processor.The processor receives monitoring unit data that includes images fromthe camera and sensor data from one or more sensors. The system includesa server located remote to the premises and coupled to the monitoringunit via the network interface. The server is coupled to a database. Theserver receives monitoring unit data from the monitoring unit and storesthe monitoring unit data in the database. The server provides access tothe monitoring unit data via a mobile device associated with a user.

Embodiments described herein include a system for remote monitoring, thesystem comprising: a monitoring unit at a premises, the monitoring unitcomprising a processor coupled to a camera and a network interface,wherein the processor includes at least one application executing on theprocessor, wherein the processor receives monitoring unit data thatincludes images from the camera and sensor data from one or moresensors; and a server located remote to the premises and coupled to themonitoring unit via the network interface, wherein the server is coupledto a database, wherein the server receives monitoring unit data from themonitoring unit and stores the monitoring unit data in the database,wherein the server provides access to the monitoring unit data via amobile device associated with a user.

Embodiments described herein include a system for remote monitoring. Thesystem comprises a monitoring unit at a premises. The monitoring unitcomprises a processor coupled to a plurality of sensors. The pluralityof sensors includes an image sensor. The processor includes at least oneapplication executing on the processor. The processor receives sensordata from the plurality of sensors and generates monitoring unit data.The system includes a server and a database located remote to thepremises and coupled to the monitoring unit via a network coupling. Theserver receives the sensor data and the monitoring unit data and storesthe sensor data and the monitoring unit data in the database. The serverprovides access to the sensor data and the monitoring unit data via amobile device.

Embodiments described herein include a system for remote monitoring, thesystem comprising: a monitoring unit at a premises, the monitoring unitcomprising a processor coupled to a plurality of sensors, wherein theplurality of sensors include an image sensor, wherein the processorincludes at least one application executing on the processor, whereinthe processor receives sensor data from the plurality of sensors andgenerates monitoring unit data; and a server and a database locatedremote to the premises and coupled to the monitoring unit via a networkcoupling, wherein the server receives the sensor data and the monitoringunit data and stores the sensor data and the monitoring unit data in thedatabase, wherein the server provides access to the sensor data and themonitoring unit data via a mobile device.

Although certain methods, apparatus, computer readable memory, andarticles of manufacture have been described herein, the scope ofcoverage of this disclosure is not limited thereto. To the contrary,this disclosure covers all methods, apparatus, computer readable memory,and articles of manufacture fairly falling within the scope of theappended claims either literally or under the doctrine of equivalents.

Although the following discloses example methods, system and apparatusincluding, among other components, software executed on hardware, itshould be noted that such methods, system and apparatus are merelyillustrative and should not be considered as limiting. For example, itis contemplated that any or all of these hardware and softwarecomponents could be embodied exclusively in hardware, exclusively insoftware, exclusively in firmware, or in any combination of hardware,software, and/or firmware. Accordingly, while the following describesexample methods and apparatus, persons having ordinary skill in the artwill readily appreciate that the examples provided are not the only wayto implement such methods, system and apparatus.

Computer networks suitable for use with the embodiments described hereininclude local area networks (LAN), wide area networks (WAN), Internet,or other connection services and network variations such as the worldwide web, the public internet, a private internet, a private computernetwork, a public network, a mobile network, a cellular network, avalue-added network, and the like. Computing devices coupled orconnected to the network may be any microprocessor controlled devicethat permits access to the network, including terminal devices, such aspersonal computers, workstations, servers, mini computers, main-framecomputers, laptop computers, mobile computers, palm top computers, handheld computers, mobile phones, TV set-top boxes, or combinationsthereof. The computer network may include one of more LANs, WANs,Internets, and computers. The computers may serve as servers, clients,or a combination thereof.

The embodiments described herein can be a component of a single system,multiple systems, and/or geographically separate systems. Theembodiments described herein can also be a subcomponent or subsystem ofa single system, multiple systems, and/or geographically separatesystems. The embodiments described herein can be coupled to one or moreother components (not shown) of a host system or a system coupled to thehost system.

One or more components of the embodiments described herein and/or acorresponding system or application to which the embodiments describedherein is coupled or connected includes and/or runs under and/or inassociation with a processing system. The processing system includes anycollection of processor-based devices or computing devices operatingtogether, or components of processing systems or devices, as is known inthe art. For example, the processing system can include one or more of aportable computer, portable communication device operating in acommunication network, and/or a network server. The portable computercan be any of a number and/or combination of devices selected from amongpersonal computers, personal digital assistants, portable computingdevices, and portable communication devices, but is not so limited. Theprocessing system can include components within a larger computersystem.

The processing system of an embodiment includes at least one processorand at least one memory device or subsystem. The processing system canalso include or be coupled to at least one database. The term“processor” as generally used herein refers to any logic processingunit, such as one or more central processing units (CPUs), digitalsignal processors (DSPs), application-specific integrated circuits(ASIC), etc. The processor and memory can be monolithically integratedonto a single chip, distributed among a number of chips or components,and/or provided by some combination of algorithms. The methods describedherein can be implemented in one or more of software algorithm(s),programs, firmware, hardware, components, circuitry, in any combination.

The components of any system that includes the embodiments describedherein can be located together or in separate locations. Communicationpaths couple the components and include any medium for communicating ortransferring files among the components. The communication paths includewireless connections, wired connections, and hybrid wireless/wiredconnections. The communication paths also include couplings orconnections to networks including local area networks (LANs),metropolitan area networks (MANs), wide area networks (WANs),proprietary networks, interoffice or backend networks, and the Internet.Furthermore, the communication paths include removable fixed mediumslike floppy disks, hard disk drives, and CD-ROM disks, as well as flashRAM, Universal Serial Bus (USB) connections, RS-232 connections,telephone lines, buses, and electronic mail messages.

Aspects of the embodiments described herein and corresponding systemsand methods described herein may be implemented as functionalityprogrammed into any of a variety of circuitry, including programmablelogic devices (PLDs), such as field programmable gate arrays (FPGAs),programmable array logic (PAL) devices, electrically programmable logicand memory devices and standard cell-based devices, as well asapplication specific integrated circuits (ASICs). Some otherpossibilities for implementing aspects of the embodiments describedherein and corresponding systems and methods include: microcontrollerswith memory (such as electronically erasable programmable read onlymemory (EEPROM)), embedded microprocessors, firmware, software, etc.Furthermore, aspects of the embodiments described herein andcorresponding systems and methods may be embodied in microprocessorshaving software-based circuit emulation, discrete logic (sequential andcombinatorial), custom devices, fuzzy (neural) logic, quantum devices,and hybrids of any of the above device types. Of course the underlyingdevice technologies may be provided in a variety of component types,e.g., metal-oxide semiconductor field-effect transistor (MOSFET)technologies like complementary metal-oxide semiconductor (CMOS),bipolar technologies like emitter-coupled logic (ECL), polymertechnologies (e.g., silicon-conjugated polymer and metal-conjugatedpolymer-metal structures), mixed analog and digital, etc.

It should be noted that any system, method, and/or other componentsdisclosed herein may be described using computer aided design tools andexpressed (or represented), as data and/or instructions embodied invarious computer-readable media, in terms of their behavioral, registertransfer, logic component, transistor, layout geometries, and/or othercharacteristics. Computer-readable media in which such formatted dataand/or instructions may be embodied include, but are not limited to,non-volatile storage media in various forms (e.g., optical, magnetic orsemiconductor storage media) and carrier waves that may be used totransfer such formatted data and/or instructions through wireless,optical, or wired signalling media or any combination thereof. Examplesof transfers of such formatted data and/or instructions by carrier wavesinclude, but are not limited to, transfers (uploads, downloads, e-mail,etc.) over the Internet and/or other computer networks via one or moredata transfer protocols (e.g., HTTP, FTP, SMTP, etc.). When receivedwithin a computer system via one or more computer-readable media, suchdata and/or instruction-based expressions of the above describedcomponents may be processed by a processing entity (e.g., one or moreprocessors) within the computer system in conjunction with execution ofone or more other computer programs.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense as opposed to anexclusive or exhaustive sense; that is to say, in a sense of “including,but not limited to.” Words using the singular or plural number alsoinclude the plural or singular number respectively. Additionally, thewords “herein,” “hereunder,” “above,” “below,” and words of similarimport, when used in this application, refer to this application as awhole and not to any particular portions of this application. When theword “or” is used in reference to a list of two or more items, that wordcovers all of the following interpretations of the word: any of theitems in the list, all of the items in the list and any combination ofthe items in the list.

The above description of embodiments and corresponding systems andmethods is not intended to be exhaustive or to limit the systems andmethods to the precise forms disclosed. While specific embodiments of,and examples for, the embodiments and corresponding systems and methodsare described herein for illustrative purposes, various equivalentmodifications are possible within the scope of the systems and methods,as those skilled in the relevant art will recognize. The teachings ofthe embodiments described herein and corresponding systems and methodsprovided herein can be applied to other systems and methods, not onlyfor the systems and methods described above.

The elements and acts of the various embodiments described above can becombined to provide further embodiments. These and other changes can bemade to the integrated security system and corresponding systems andmethods in light of the above detailed description.

In general, in the following claims, the terms used should not beconstrued to limit the integrated security system and correspondingsystems and methods to the specific embodiments disclosed in thespecification and the claims, but should be construed to include allsystems that operate under the claims. Accordingly, the integratedsecurity system and corresponding systems and methods is not limited bythe disclosure, but instead the scope is to be determined entirely bythe claims.

While certain aspects of the embodiments described herein andcorresponding systems and methods are presented below in certain claimforms, the inventors contemplate the various aspects of the embodimentsdescribed herein and corresponding systems and methods in any number ofclaim forms. Accordingly, the inventors reserve the right to addadditional claims after filing the application to pursue such additionalclaim forms for other aspects of the embodiments described herein andcorresponding systems and methods.

1. (canceled)
 2. A computer-implemented method comprising: receiving, bya user device and from a monitoring system configured to transmit videodata (a) without triggering an alarm condition at a property that alertsa security system and (b) in response to determining that sensor datacollected by one or more sensors located at the property is (i)logically related to a notification-only rule that when satisfiedtriggers data transmission to a device associated with the property and(ii) not logically related to one or more security rules that whensatisfied triggers data transmission to the security system, data for avideo stream; in response to receiving the video stream, generating auser interface that includes a video presentation field for presentationof content from the video stream and one or more controls that enableinteraction with content depicted in the video presentation field; andcausing, using at least some of the data for the video stream,presentation of the user interface on a display.
 3. Thecomputer-implemented method of claim 2, comprising: receiving at leastone of a pinching gesture or a swiping gesture to activate one of theone or more controls and in response to receiving at least one of thepinching gesture or the swiping gesture, adjusting presentation of datafor the video stream in the video presentation field of the userinterface.
 4. The computer-implemented method of claim 2, whereincausing presentation of the user interface comprises causingpresentation of the user interface that presents, in the videopresentation field, a plurality of angles of the video stream in asegmented view.
 5. The computer-implemented method of claim 2,comprising: receiving, through a second user interface, user inputdefining one or more parameters for the notification-only rule that,when satisfied, triggers data transmission to a device associated withthe property and does not trigger data transmission to the securitysystem; and providing, to the monitoring system, data a) defining thenotification-only rule b) that includes the one or more parameters. 6.The computer-implemented method of claim 2, wherein receiving the datafor the video stream comprises: receiving lens-warped video data for thevideo stream captured with a camera with a wide field-of-view lens;de-warping the lens-warped video data to create de-warped video data;and presenting, in the user interface, the de-warped video data.
 7. Thecomputer-implemented method of claim 6, wherein de-warping thelens-warped video data comprises: mapping the lens-warped video data toa three-dimensional space that replicates a geometry of the widefield-of-view lens; and de-warping, using the mapping of the lens-warpedvideo data to the three-dimensional space, the lens-warped video data.8. The computer-implemented method of claim 6, wherein de-warping thelens-warped video data comprises: creating, using an applicationprogramming interface for rendering vector graphics, a virtual surface;and de-warping, using the virtual surface, the lens-warped video data.9. A system comprising one or more computers and one or more storagedevices on which are stored instructions that are operable, whenexecuted by the one or more computers, to cause the one or morecomputers to perform operations comprising: receiving, by a user deviceand from a monitoring system configured to transmit video data (a)without triggering an alarm condition at a property that alerts asecurity system and (b) in response to determining that sensor datacollected by one or more sensors located at the property is (i)logically related to a notification-only rule that when satisfiedtriggers data transmission to a device associated with the property and(ii) not logically related to one or more security rules that whensatisfied triggers data transmission to the security system, data for avideo stream; in response to receiving the video stream, generating auser interface that includes a video presentation field for presentationof content from the video stream and one or more controls that enableinteraction with content depicted in the video presentation field; andcausing, using at least some of the data for the video stream,presentation of the user interface on a display.
 10. The system of claim9, the operations comprising: receiving at least one of a pinchinggesture or a swiping gesture to activate one of the one or more controlsand in response to receiving at least one of the pinching gesture or theswiping gesture, adjusting presentation of data for the video stream inthe video presentation field of the user interface.
 11. The system ofclaim 9, wherein causing presentation of the user interface comprisescausing presentation of the user interface that presents, in the videopresentation field, a plurality of angles of the video stream in asegmented view.
 12. The system of claim 9, the operations comprising:receiving, through a second user interface, user input defining one ormore parameters for the notification-only rule that, when satisfied,triggers data transmission to a device associated with the property anddoes not trigger data transmission to the security system; andproviding, to the monitoring system, data a) defining thenotification-only rule b) that includes the one or more parameters. 13.The system of claim 9, wherein receiving the data for the video streamcomprises: receiving lens-warped video data for the video streamcaptured with a camera with a wide field-of-view lens; de-warping thelens-warped video data to create de-warped video data; and presenting,in the user interface, the de-warped video data.
 14. The system of claim13, wherein de-warping the lens-warped video data comprises: mapping thelens-warped video data to a three-dimensional space that replicates ageometry of the wide field-of-view lens; and de-warping, using themapping of the lens-warped video data to the three-dimensional space,the lens-warped video data.
 15. The system of claim 13, whereinde-warping the lens-warped video data comprises: creating, using anapplication programming interface for rendering vector graphics, avirtual surface; and de-warping, using the virtual surface, thelens-warped video data.
 16. A non-transitory computer storage mediumencoded with instructions that, when executed by one or more computers,cause the one or more computers to perform operations comprising:receiving, by a user device and from a monitoring system configured totransmit video data (a) without triggering an alarm condition at aproperty that alerts a security system and (b) in response todetermining that sensor data collected by one or more sensors located atthe property is (i) logically related to a notification-only rule thatwhen satisfied triggers data transmission to a device associated withthe property and (ii) not logically related to one or more securityrules that when satisfied triggers data transmission to the securitysystem, data for a video stream; in response to receiving the videostream, generating a user interface that includes a video presentationfield for presentation of content from the video stream and one or morecontrols that enable interaction with content depicted in the videopresentation field; and causing, using at least some of the data for thevideo stream, presentation of the user interface on a display.
 17. Thecomputer storage medium of claim 16, the operations comprising:receiving at least one of a pinching gesture or a swiping gesture toactivate one of the one or more controls and in response to receiving atleast one of the pinching gesture or the swiping gesture, adjustingpresentation of data for the video stream in the video presentationfield of the user interface.
 18. The computer storage medium of claim16, wherein causing presentation of the user interface comprises causingpresentation of the user interface that presents, in the videopresentation field, a plurality of angles of the video stream in asegmented view.
 19. The computer storage medium of claim 16, theoperations comprising: receiving, through a second user interface, userinput defining one or more parameters for the notification-only rulethat, when satisfied, triggers data transmission to a device associatedwith the property and does not trigger data transmission to the securitysystem; and providing, to the monitoring system, data a) defining thenotification-only rule b) that includes the one or more parameters. 20.The computer storage medium of claim 16, wherein receiving the data forthe video stream comprises: receiving lens-warped video data for thevideo stream captured with a camera with a wide field-of-view lens;de-warping the lens-warped video data to create de-warped video data;and presenting, in the user interface, the de-warped video data.
 21. Thecomputer storage medium of claim 20, wherein de-warping the lens-warpedvideo data comprises: mapping the lens-warped video data to athree-dimensional space that replicates a geometry of the widefield-of-view lens; and de-warping, using the mapping of the lens-warpedvideo data to the three-dimensional space, the lens-warped video data.22. The computer storage medium of claim 20, wherein de-warping thelens-warped video data comprises: creating, using an applicationprogramming interface for rendering vector graphics, a virtual surface;and de-warping, using the virtual surface, the lens-warped video data.